Data Fusion and Object Identification

Many databases exist, which are defined on the same universe of objects, but there is no global ID available at all. To fuse data it is necessary to identify objects. As an example think of several population databases, where only name and address are common attributes. Data quality varies over the sources. Therefore the identification is more complicated due to erroneous and obsolete data. The usual approach of data integration is to establish a global schema, which is derived from the local schemata. In the case of missing ID’s the global schema is not sufficient for identification. Hence we compute appropriate derivable attributes. Records are compared pairwise based on those derived attributes. The underlying comparison space is partitoned into at least two classes same and not same. To classify records according to these comparison values, supervised learning can be applied. Data Fusion is more than integration because it merges data and evaluates the quality of the merging. We develop a unified framework for such kind of data fusion. We cover the feature selection problem, and embed the data fusion problem into supervised classification. This approach makes it possible to select the best procedure for data fusion and object identification from solid methodologies like likelihood ratio tests (record linkage), classification trees or non linear classification. Unfortunately, for supervised learning a random sample of pairs must be taken and each pair labeled. Alternatively, a weighted similarity measure can be defined and used for classification. Referring to modern theory of distributed, heterogeneous databases we add a third component to the three-layer Mediator-Wrapper architecture of data integration introduced by Wiederhold: The Identificator. We illustrate the approach by an example.